Learning Objectives

After completing this lesson, you’ll be able to:

• Describe why debugging is an important skill.
• Distinguish between different types of log messages.
• Find the text and spatial log files.
• Explain the difference between errors and warnings.

Instructions

In this lesson, you will:

• Optional: Watch the Use the Log to Identify Problems video.
• Scroll down to read the text below.
• Optional: Let us know if you found this lesson relevant to your role by filling out the survey at the bottom of the page.
• Click 'Next' to mark the lesson complete.

Resources

• Starting workspace 
  • For Safe Software-hosted training courses, you can find this on your virtual machine here: C:\FMEData\Workspaces\UseDataIntegrationBestPractices\use-the-log-to-identify-problems.fmw
• Complete workspace
  • C:\FMEData\Workspaces\UseDataIntegrationBestPractices\use-the-log-to-identify-problems-complete.fmw
• Addresses.gdb.zip (File GDB)
  • C:\FMEData\Data\Addresses\Addresses.gdb

Debugging

In this course, we’ll cover best practices for debugging your workspaces.

Even skilled FME users seldom produce new workspaces with zero defects. For that reason, all users must be aware of the debugging techniques available in FME.

Generally, debugging in FME consists of determining whether a problem has occurred and then tracking down the problem's source (for example, where it appears in the workspace). Once you have determined where a problem arises, you can investigate it.

There are various debugging techniques available in FME, and it's essential to use these in the correct order. For example, you shouldn't waste time randomly changing parameters and re-running the workspace when a simple log message explains the issue!

A logical order would be:

• Check for any problems
  • Interpret the log for warnings and errors
  • Inspect the output datasets
• Locate the problem
  • Review connection record counts
  • Inspect the data at key stages of the translation
• Determine the problem
  • Check reader, writer, or transformer parameters at the point of failure
  • If necessary, run the workspace in feature debug mode

In software development, debugging a process after completion is called "post-mortem debugging"! We're trying to find out what caused a fatality.

Log Interpretation

FME logs contain a record of all stages and processes within a translation. The contents are vital for debugging purposes.

Log Message Types

Different message types appear in the log window, including:

• Error
  • An error, denoted in the log by red text and the term ERROR, indicates that a problem has caused FME to cease processing. For example, FME cannot write the output dataset because of incorrect user permissions.
• Warning
  • A warning, denoted by blue text and the term WARN, indicates a processing problem. The problem is sufficiently minor to allow FME to complete the translation, but you may want to check if the output was adversely affected. For example, FME cannot write features because its geometry is incompatible with the writer format. FME Workbench will drop the features from the translation and issue a warning in the log.
• Information
  • Information messages, denoted by the term INFORM, indicate information that may help users determine whether their translation functioned correctly. For example, FME sometimes logs confirmation of a particular dataset parameter, such as the coordinate system.
• Statistics
  • Statistics messages, denoted by the term STATS, provide information on various numbers relating to the translation, such as the number of features FME read from a source dataset and the time it took.

The Log window is displayed as a table. The Transformer column shows you which transformer an error/information message is coming from. Click the hyperlinked transformer name to navigate to the canvas element producing the message. Using this link to identify where errors are occurring will make your debugging more efficient.

Spatial Log File

Besides writing the log to a text file (<workspace name>.log), FME also writes a spatial log:

The spatial log is a dataset of features (in FME Feature Store format) that the log mentions. FME generates a spatial log from <Rejected> Features when Navigator > Workspace Parameters > Reader/Writer Redirect is enabled or when a Logger has Feature Logging set to "Log and Record." You can open the FFS dataset within FME Data Inspector or the Data Preview window in FME Workbench to inspect the features and identify any problems that caused FME to reject them.

Interpreting the Log Window

The Translation Log window should be the first place you check when a translation finishes. It will tell you if there are any concerning errors or warnings.

Errors

If an ERROR occurs, FME will likely stop the translation. There will be red text in the log and some statements such as:

Program Terminating
Translation FAILED.

There may be several ERROR messages, so scroll back up the log window and identify the first of these, which is the likely cause of the problem. For example, this message:

ERROR |Error connecting to PostgreSQL database(host='postgis.train.safe.com', port='5432', dbname='fmedata', user='fmedata', password='*'): 'FATAL: password authentication failed for user "fmedata" FATAL: password authentication failed for user "fmedata"

...is an obvious problem authenticating a database connection.

Warnings

Even when a translation succeeds, it's essential to check the log for the following comment:

Translation was SUCCESSFUL with X warning(s)

If there are any warnings (for example, if X > 0), use the search option to look for the word WARN. Any warning messages might significantly affect the quality of the output data.

Finding the Error's Source

Click the log hyperlink to look at the object on the canvas that produced the error.

Exercise

Frank is working on a workspace that reads address data from a File Geodatabase and writes it to a CSV with a new schema. He notices that text attributes containing Traditional Chinese characters are rendered incorrectly in the output. For example, the name "麥毅倫" appears as "???." He suspects a text encoding issue and needs to use his debugging skills at critical stages to determine when the data last looked correct.

In this exercise, you will:

• Inspect data at multiple stages to identify where an encoding problem occurs.

1) Open and Run Starting Workspace

• Start FME Workbench (2026.1 or later).
• Open your starting workspace: (C:\FMEData\Workspaces\UseDataIntegrationBestPractices\use-the-log-to-identify-problems.fmw).
• Click Run.

2) Inspect the Source Data Before Reading

It's easy to assume source data is correct without checking it. Before tracing the encoding problem in FME, confirm that the data looks correct in its native application.

• Open the Addresses.gdb dataset in ArcGIS (or ArcCatalog).
  • If you are taking a Safe Software-hosted training course, ArcCatalog and ArcGIS Pro are not installed; refer to the image below instead.
•  Confirm that the OWNERNM2 attribute displays Chinese characters correctly.
  • If the source data is already incorrect, the problem is not in FME.

3) Inspect the Source Data After Reading

Now confirm that FME reads the source data correctly. One of the features with a Chinese character value belongs to owner Marisa Marmol. You'll use her name to filter down to the problem feature.

• In the FeatureReader, click the PostalAddress cache.

• In the search bar, enter: Marmol.
  
  • This will filter to the problem feature.

• Observe that the OWNERNM2 attribute still displays Traditional Chinese characters correctly.
  • The Chinese characters should display correctly at this stage.
  • If they don't, the encoding problem was introduced during reading.

4) Inspect the Data Before Writing

Now check whether the data is still correct after transformation and before writing.

• On the AttributeManager, click the Output cache.

• Search for Marmol again.
• Observe that the Owner2 attribute displays Chinese characters correctly.
  • The Chinese characters should still be correct here.
  • If they appear as replacement characters (�), you could assume the problem occurred during transformation before writing and could use data caching to inspect each step and narrow down where it appears.

5) Inspect the Output Dataset in Data Preview

Since the data is correct going into the writer, check whether the problem occurs during writing.

• Click the PostalAddress writer feature type.
• Click View Written Data.

• Search for Marmol.
• Observe that the Owner2 attribute no longer displays Chinese characters correctly.
  • This time, the Chinese characters in the Owner2 attribute are rendered as ???.
  • Because the data was correct before writing but incorrect after, FME incorrectly encoded the data during writing.

6) Inspect the Output in a Text Editor

Opening the output file in a text editor is another useful check. It is not possible to do this for every dataset, such as binary files or databases, but text-based files can confirm whether the data is correct in the destination.

• Open the output PostalAddress.csv in Notepad (or any text editor). 
• Search for Marmol.
• Confirm if the ??? characters appear in the written file.
  • Your machine might show an empty string instead of ???.
  • If the Owner2 value appears as ???, the encoding problem was written into the file. If it displays correctly, the issue may be with how another application is interpreting the file.

7) Inspect the Output Dataset in Another Application

Although not the case in our scenario, if the file appears correct in a text editor but still displays incorrectly in the destination application, the problem is likely with the application's interpretation of the encoding.

• Open the output PostalAddress.csv in Microsoft Excel.
  • If you are attending a Safe Software-hosted training, open the file in Open Office Calc or review the image below.
• Search for Marmol and check the Owner2 column.
  
  • If ??? is still visible here, this confirms the encoding problem was written into the file by FME. 

Inspecting data at multiple stages helps you narrow down where an encoding problem occurs, but it does not always reveal the root cause. The problem could be a writer parameter set incorrectly, a limitation of the writer format, or a difference in how two applications interpret the same file. Once you know which stage introduced the error, you have a clear starting point for investigating why.

Leave Us Feedback on This Lesson